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Abstract 

Consider a nonlinear regression model ■ yi = g (xi, 0) + ei, i = 1, n, 
where the Xi are random predictors Xi and 6 is the unknown parameter 
vector ranging in a set C-R''. All known results on the consistency of 
the least squares estimator and in general of M estimators assume that 
either is compact or g is bounded, which excludes frequently employed 
models such as the Michaelis-Menten, logistic growth and exponential 
decay models. In this article we deal with the so-called separable models, 
where p = pi + p2, 6 ^ {a, P) with a eA C R^^ , l3 eB C R^^'and g 
has the form g(x,6) — /3^h(x,a) where h is a function with values in 
R^^ . We prove the strong consistency of M estimators under very general 
assumptions, assuming that h is a bounded function of a, which includes 
the three models mentioned above. 

Key words and phrases: Nonlinear regression, separable models, con- 
sistency, robust estimation. 
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1 Introduction 



Consider i.i.d. observations (xi,yi) , i — 1, ...,n, given by the nonlinear model 
with random predictors: 

yi = g{^i,Oo) + e„ (1) 

where e R"^ and are independent, and the unknown parameter vector 
ranges in a set Q C R^. An important case, usually called separable, are models 
where p = pi + P2 and 6*0= (ao, /Jg) with ao&A C R^^ and Po&B C R^^ , and g 
of the form 

P2 

g (x, e)=g (x, «, /3) = 5] (x, a) , (2) 

i=i 

where hj (j = 1, ...,^2) are functions of X x Rp^ — > i?. Usually B is the whole of 
RP^ or an unbounded subset of it. Examples are the Michaelis-Menten model, 
with 

X 

Pi ~ P2 = Q = ^, X > 0, a, /? > 0, h(x,a) — , (3) 

X + a 

the logistic growth model, with 

q = 1, p2 = I, P2 = 1, > 0, a," > 0, (3 > 0, h (x, a) ~ — 

1 + ai (e 



(e"2^ - 1) 

(4) 

the exponential decay model, with 

pi 

g = 1, P2 - Pi + 1, a: > 0, a, < 0, /?, > 0, g (x, a, /3) = /3o + '^J^"^"' (5) 

and the exponential growth model, like ([5]) but with aj > 0. 
The classical least squares estimate (LSE) is given by 
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The consistency of the LSE assuming E(ej) = and Var(ej) = tr^ < oo 
has been proved by several authors under the assumption of a compact 0; in 
particular Amemiya (1983), Jennrich (1969) and Johansen (1984). Wu (1981) 
assumes that 6 is a finite set. 

Richardson and Bhattacharyya (1986) do not require the compactness of 0, 
but they assmiic g (x, 0) to be a bounded function of 0, which excludes most 
separable models. 

Shao (1992) showed the consistency of the LSE without requiring the com- 
pacity of nor the boundedness of g, but requires assumptions on g that exclude 
the simplest separable models. For example, in the case g {x, 6) = ^e"^, for any 
a;o > one can make g {xq, 9) =constant with a — oo and ^ — >■ 0. This fact 
violates both "Condition 1" and "Condition 2" in page 427 of his paper. 

The well-known fact that the LSE is sensitive to outliers has led to the devel- 
opment of robust estimates that are simultaneously highly efficient for normal 
errors and resistant to perturbations of the model. One of the most important 
families of robust estimates are the M-estimates proposed by Huber (1973) for 
the linear model. For nonlinear models they are defined by 



where p is a loss function whose properties will be described in the next section 
and a is an estimate of the error's scale. However, at this stage of our research 
we deal with the simpler case of known a. Then it may be assumed without loss 
of generality that a = 1 and therefore we shall deal with estimates of the form 



All published results on the consistency of robust estimates for nonlinear 





n 
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models require the compacity of 6. Oberhofer (1982) deals with the Li esti- 
mator. Vainer and Kukush (1998) and Liese and Vajda (2003, 2004) deal with 
M estimates. The latter deal with O (n~^/^) consistency and asymptotic nor- 
mality of M estimates in more general models. Stromberg (1995) proved the 
consistency of the Least Median of Squares estimate (Rousseeuw, 1984), and 
Cfzok (2005) dealt with the consistency and asymptotic normality of the Least 
Trimmed Squares estimate. Fasano et al. (2012) study the functionals related 
to M estimators in linear and nonlinear regression; in the latter case, they also 
assume a compact 0. 

In this article we will prove the consistency of M estimates for separable mod- 
els without assuming the compactness of O, but assuming the boundedness of 
the hjs; this case includes the exponential decay, logistic growth and Michaelis- 
Menten models. It can thus be considered as a generalization of (Richardson 
and Bhattacharyya, 1986). 



2 The assumptions 

It will be henceforth assumed that p is a "p-function" in the sense of (Maronna 
et al, 2006). i.e., p{u) is a continuous nondecreasing function of such that 
p{0) =0 and that if p{u) < sup„p(w) and < u < v then p{u) < p{v). We 
shall consider two cases: unbounded p and bounded p. The first includes convex 
function, in particular the LSE with p{x) = and the well-known Huber 
function 

if \x\ < k 



Pk{x) 



(8) 

2k\x\-k'^ if |a;|>A; 



and the second includes the bisquare function p (x) = min ~ (^/^)^) > ^ | > 

where k is in both cases a constant that controls the estimator's efficiency. 
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Let h (x, a) — (/ii (x, a) , /ipj (x, a))' where in general a' denotes the 
transpose of a. The necessary assumptions are: 

A B is a closed set such that tf3 eB for all /? and t > 0. 
B supQ,g^E|p(?/ -/3'h(x, a)) I < oo for all (3 eB. 

C The function Ep (e — t) -where e denotes any copy of e^- has a unique min- 
imum at t — 0. Put Aq = Ep (e) . 

D h is continuous in a a.s. and 

a ^ ao ^ sup P {/3'h (x, a) = /3> (x, a^)} < 1 (9) 

E Let S = supi /9 (t) (which may be infinite) . Then 

d =: sup P (^'h (x, a) 0) < 1 

F Call U the family of all open neighborhoods of oq. Then 

sup inf sup P {/3'h (x, a) = /Jph (x, ag)} < 1. 

G h is bounded as a function of a, i.e., sup^g^ l|h (x, a)l| < oo a.s. 
We now comment on the assumptions. 

For (A) to hold in examples ©-(HI)-® we must enlarge the range of /3jS to 
Pj > 0. However, to ensure the validity of (D) and (F), it will be assumed that 
the elements of the "true" vector /3o are all positive. 

If p is bounded, (B) holds without further conditions. Sufficient conditions 
for Huber's p and for the LSE are finite moments of e and of h (x, a) , of orders 
one and two, respectively. 



Ao 

s ■ 



(10) 
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A sufficient condition for (C) is tliat the distribution of e has an even density 
/ (u) that is nonincreasing for u > and is decreasing in a neighborhood of u = 
(see Lemma 3.1 of Yohai (1987)). If p is strictly convex with a derivative V') 
then a sufficient condition is Eip (e) = 0, which for the LSE reduces to Ee — 0. 

Assumption (D) is required for ensure uniqueness of solutions. For examples 
©-(HI) it is very easy to verify. For ([S|) it follows from the well-known linear 
independence of exponentials. 

If 5 = oo, (E) just means that S < 1 (since Aq < oo by (B)). Otherwise it 
puts a bound on S. In our examples we have ^ = 0, since /3'h > if /3 has a 
single nonnull (positive) element. 

Assumption (F) is required in the case of non-compact A, to prevent the 
estimator a from "escaping to the border" . In our examples the border for 
the ajS is either zero of infinity, and (F) is easily verified by a detailed but 
elementary calculation (taking into account the remark above that all elements 
of /3o are positive) . For example, in Q it suffices to consider neighborhoods of 
the form {aQ/K,Kao) with K sufficiently large. 

Finally, (G) is easily verified for models ©-(HI)-®. 



We now state our main result. 

Theorem 1 Assume model with conditions A-B-C-D-E-F-G. Then the M 
estimate (Sn,/3n) is strongly consistent for Oq. 



3 The results 



For separable models the M-estimate is given by 




1=1 
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We shall first need an auxiliary result, based on a proof in (Bianco and 
Yohai, 1996). 

Lemma 2 Assume model with conditions A-B-C-D-E and A compact. Then 
/3„ is ultimately bounded with probability one. 

Proof of the Lemma: Put 

A(a,/3) = Ep(y-/3'h(x,a)). 



It follows from (C) that X{a, (3) attains its minimum only when /3'h (x, a) = 
/3oh(x, ag) a.s. and by © this happens when (a,/3) = (ao,/3o) ■ Therefore 

(a, (3) ^ (ao, /3o) ^ A (a, /3) > A (ao, fio) = Aq. (11) 

Let r = {7 e B : II7II = 1} . Then we may write /3 = ^7 with t = ||^|| e R+ 
and 7 G r. 

We divide the proof into two cases. 

Case I: bounded p : Assume that S — sup„ p [u) < 00. To simplify notation 
it will be assumed without loss of generality that S = 1. For each (a, 7) e A x F 
we have 

lim Ep {y-t-f'h (x, a)) >l-5> Aq, 
where S is defined in (fTOl). Let 



^ = l-<5-Ao>0, £ = i<i^. 



Since ([TO]) implies that P (|7'h (x, a)| > 0) > 1 - (5 for 7 eF, then for each 
(a, 7) G A X F there are positive a, b such that 

Pdyl <a,|7'h(x,a)| >6) >l~S~e. (12) 
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Then by ((H]) there exists T > such that t>T impHes 



E inf p(y-i7'h(x,a)) >1-S~2e. 



(13) 



Therefore ([T^ impUcs that for each (a, 7) G ^xF there exist a neighborhood 
/7 (a, 7) C A X r and T (a, 7) G i?+ such that 

E inf inf p{y-W,h{^,a,))>l-6-2e = Xo + ^. (14) 

(ai,7i)eC/(Q,7) t>T{a,-f) 2 

The neighborhoods {C/(a,7) : a €^,7 G F} are a covering of the compact 
set ^xF, and therefore there exists a finite subcovering thereof: {Uj = U {ctj,^j)}'jLi- 
Let Tq — maxj T [aj ,"fj) . 

We shaU show that hnisup„_>oQ /3„ < Tq a.s. Put for brevity 



1 " 

A„ (a, /3) = - p (y^ - /3'h (a;^, a)) 



Then 



1 " 

inf inf A„ (a, 13) > — > inf inf p (w,- — t7'h (a;,, a)) 

1 " 

= min — > inf inf p (?/, — i7'h (a;, , a)) 

j = l,...,Wn^(a,7)GC/,t>To'^^^* ' ^ 

and therefore (|14p and the Law of Large Numbers imply 



while 



lim inf inf inf A„ (a, /?) > Aq H — a.s. 

n^oo ||/3||>To qGA 2 



A„ (a„,/3„ ) = inf inf A„ (a,/3) < A„ (ao,/?o) -^o a.s. 



which shows that ultimately /3„ < Tq with probability one 
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Case II: unbounded p : Here an analogous but simpler procedure shows 
the existence of To and neighborhoods U (a, 7) such that the left-hand member 
of (|14p is larger than 2Ao, and the rest of the proof is similar. ■ 

Proof of the Theorem: If A is not compact, we employ the same approach 
as in (Richardson and Bhattacharyya, 1986): the Cech-Stone compactification 
yields a compact set A Z) A such that each bounded continuous function on 
A has a unique continuous extension to A. We have to ensure that (B), (D) 
and (E) continue to hold for a ^A. Since each element of A is the limit of a 
sequence of elements of A, (B) and (E) are immediate; and (D) follows from 
assumption (F). Therefore we can apply the Lemma to conclude that ^a„,/3„^ 
remains ultimately in a compact a.s. The Theorem then follows from Theorem 
1 of Huber (1967). ■ 
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